A Text Categorization Approach to Video Scene Classification Using Keypoint Features

Scene classification based on local keypoint features has emerged as a promising research direction. Each image is represented by a “bag of visual words” as high-dimensional, vector-quantized keypoint features, which is analogous to the “bag of words” representation of text documents. Based on such representation, we take a fully text categorization approach to the scene classification problem, with an emphasis on comparatively evaluating various implementation choices related to this visual-word representation, including vocabulary size, feature weighting and normalization, feature selection, etc. We intend to practical insights as to the optimal representation choices that maximize the classification performance. Scene classification experiments based on a large video corpus lead to many important observations: 1) using our approach, carefully engineered visual-word features achieve comparable performance to that of traditional color/texture features, and their combination significantly enhances the performance; 2) the visual word distribution in the video corpus bears many similarities yet important differences to the word distribution in a text corpus; 3) a vocabulary much larger than the ones currently used is preferred; 4) frequent visual words are not “stop words” but more informative than rare words; 5) binary features are as effective as tf or tf-idf features, and normalization always hurts the classification performance; 6) feature selection can reduce the vocabulary size by half without loss of performance; 7) spatial information is much more useful with a small vocabulary than with a large one.